Collaborative Intelligent Reflecting Surface Networks With Multi-Agent Reinforcement Learning

Intelligent reflecting surface (IRS) is envisioned to be widely applied in future wireless networks. In this paper, we investigate a multi-user communication system assisted by cooperative IRS devices with the capability of energy harvesting. Aiming maximize long-term average achievable rate, an optimization problem formulated jointly designing transmit beamforming at base station (BS) and discrete phase shift IRSs, constraints on power, user data rate requirement buffer size. Considering time-varying channels stochastic arrivals harvested first formulate as Markov decision process (MDP) then develop novel multi-agent Q-mix (MAQ) framework two layers decouple parameters. The higher layer for optimizing resolutions, lower one power allocation. Since integer programming large-scale action space, improve MAQ incorporating Wolpertinger method, namely, MAQ-WP algorithm achieve sub-optimality reduced dimensions space. addition, still high complexity good performance, propose policy gradient-based algorithm, MAQ-PG, mapping actions into continuous space cost slight performance loss. Simulation results demonstrate that proposed MAQ-PG algorithms can converge faster improvements 10.7% 8.8% over conventional DDPG, respectively.